Visual content signal display apparatus and a method of displaying a visual content signal therefor

ABSTRACT

The invention relates to a visual content signal display apparatus ( 100 ) comprising a receiver ( 101 ) which receives a visual content signal. The receiver ( 101 ) is coupled to a primary display processor ( 103 ) which is coupled to a primary display ( 105 ) and presents the visual content signal thereon. The receiver ( 101 ) is further coupled to an extraction processor ( 107 ) which extracts background content information from the visual content, signal. The background content information is fed to a secondary display processor ( 109 ) which generates a surround image on the basis of the extracted background content information. The surround image comprises visual characteristics corresponding to the visual characteristics of the image of the primary display ( 105 ). The display apparatus ( 100 ) further comprises a secondary display unit ( 111 ), coupled to the secondary display processor ( 109 ), and operable to display the surround image on a secondary display area whereby a combined display having an increased viewing angle is achieved. The surround image is preferably projected onto walls and ceiling of a room thereby providing an enhanced viewing experience.

The invention relates to a visual content signal display apparatus and amethod of displaying a visual content signal therefor and in particularto a system for enhancing a primary display.

In recent years, the variety and quantity of content that may beprovided to consumers have increased significantly with more contentbeing available through an increasing number of distribution mediumsincluding for example satellite TV broadcasts, Digital Versatile Disc(DVD), cable broadcast and terrestrial TV.

In addition to the increased quantity of content and the increaseddistribution flexibility, a significant effort has been made by thecontent provision industry to improve the user experience foraudiovisual content. For example, the advent of digital videodistribution such as digital TV or DVD has resulted in a significantlyimproved quality of the pictures and sounds that can be produced in adomestic environment. Furthermore, the advent of widescreen TVs and inparticular flat screen TVs has led to an increased prevalence of largescreen TVs having aspect ratios closer resembling those found in forexample cinemas.

As another example of an increased user experience is the advent of homecinema and in particular of surround sound where a number of speakers(typically five or seven directional speakers plus a subwoofer for basseffects) are situated around a listening position thereby providing animmersed audio experience.

It has also been suggested that improved user experience may be achievedby providing a broader viewing angle for a user. For example, cinemasare known which utilise a large semi-spherical screen which partiallysurrounds the user. An example is the IMAX theatre performances.However, these are very large, expensive and impractical for mostpractical (domestic) environments.

U.S. Pat. No. 5,963,247 discloses a visual display system whereincomposite three-dimensional-like image sequences are generated byassembling two or more separate image sequences. In the describedsystem, separate image sequences are recorded and projected on todifferent screens. The screens are positioned and the separate imagesequences synchronised such that a viewer sees a composite pictureextending over the two or more screens.

However, the system of U.S. Pat. No. 5,963,247 tends to have a number ofassociated disadvantages including the following. Firstly, the system israther complex and requires processing of a plurality of separate imagesequences. This processing is required both for transmission andreception of the image sequences. Furthermore, more than one imagesequence must be recorded which requires additional equipment andintroduces additional inconveniences and tasks associated with therecording. Similarly, a plurality of image sequences must be distributedwhich increases the required bandwidth and uses additional distributionresource. This may prevent distribution in existing communicationsystems having a limited available bandwidth. Also, as conventionalvisual content signals such as e.g. films or TV programmes do notcomprise a plurality of separate correlated image sequences, theenhanced viewing experience cannot be achieved for existing contentsignals. Additionally, the system requires more than one screen as wellas an accurate relative positioning of the screens in order to achievethe required spatial synchronisation.

Hence, an improved visual content signal display system would beadvantageous and in particular a system allowing for increased userexperience, increased flexibility, reduced complexity, reduced cost,increased backwards compatibility and/or facilitating recording of acontent signal would be advantageous.

Accordingly, the Invention preferably seeks to mitigate, alleviate oreliminate one or more of the above mentioned disadvantages singly or inany combination.

According to a first aspect of the invention, there is provided a visualcontent signal display apparatus comprising: means for receiving avisual content signal; means for presenting the visual content signal ona primary display; means for extracting background content informationfrom the visual content signal; means for generating a surround image inresponse to the background content information; and means for displayingthe surround image on a secondary display area thereby providing acombined display having an increased viewing angle.

The invention allows for an increased user experience by providing acombined display with an increased viewing angle. For example, thesecondary display area may partially or substantially or even fullysurround one or more viewers thereby providing a more absorbing viewingexperience. The viewing experience may provide a surround imageexperience compatible with a surround sound experience thereby providinga truly immersing audiovisual experience. The secondary display areaneed not consist in one or more screens or dedicated display elementsbut may use any suitable area.

The image of a primary display may be extended by use of backgroundcontent information of a content signal being presented on the primarydisplay thereby obviating or mitigating requirements for separate imagesequences. Hence, a visual content signal display apparatus having lowcomplexity and/or high flexibility may be obtained. Specifically, adisplay apparatus suitable for domestic use and the consumer market maybe enabled or facilitated.

Furthermore, recording and distribution of signals are significantlyfacilitated as only a single content signal needs to be recorded andgenerated. Also, the visual content display apparatus may be fullybackwards compatible and may be used with existing recording, transmit,broadcasting and/or distribution means and can be used with existingcontent signals. For example, the visual content display apparatus maybe used to enhance the viewing experience of traditional TV broadcastsor DVD content.

It will be appreciated that the surround image is preferably not astatic image but a dynamic surround image sequence associated with thevisual content signal displayed on the primary display. The term imageis used to include both singular static images as well as imagesequences including for example video clips. Thus an image may comprisea single picture or may comprise a plurality of subsequent pictures e.g.providing a video sequence.

According to a feature of the invention, the means for displaying thesurround image is operable to project the surround image onto anexternal surface of an object.

The secondary display area may be an external surface of a suitableobject and specifically may not be an object dedicated for presentingthe surround image. Rather the surround image may be projected onavailable surfaces thereby reducing cost, complexity and/or size of thevisual content display apparatus. Furthermore, it mitigates anyrequirements for dedicated screens or setups required for displaying thesurround image and thus facilitates deployment.

According to a different feature of the invention, the external surfaceis an internal surface of a room. This provides for a particularlysimple visual content display apparatus providing an enhanced viewingexperience. The viewing experience may be improved as external surfacesof a room provide a suitable source for displaying a surround image andparticularly for providing a surround experience. It furthermorefacilitates deployment. The internal surface may preferably be one, moreor all of the floor, ceiling and walls of a room.

According to a different feature of the invention, the means forextracting is operable to extract real time background contentinformation from the visual content signal, and the means for generatingthe surround image is operable to generate a real time surround image inresponse to the real time background content information.

The surround image may be dynamically varied to reflect the contentsignal displayed on the primary display. Thus a dynamically changingimage may be presented which extends beyond that of the primary displayand of the received visual content signal thus providing an improvedviewing experience. The surround image may specifically change withscene changes and other changes in the background of the primary displaythus providing a surround image which closely follows the image of theprimary display thereby enhancing the continuity of the combineddisplay.

According to a different feature of the invention, the means forextracting is operable to extract the background content information inresponse to background meta-data comprised in the visual content signal.The visual content signal may specifically comprise meta-data whichindicates the current background. For example, the meta-data mayindicate that the visual content signal is of a football match which maybe used to generate a surround image corresponding to e.g. a crowd ofspectators. The meta-data may also directly indicate backgroundcharacteristics or objects, e.g. the data may for example indicate thatpart of the primary display consists of a sky. This allows for aparticularly low complexity and/or accurate way of generating thebackground content information.

According to a different feature of the invention, the means forextracting is operable to extract the background content information inresponse to a content analysis of the visual content signal. This allowsfor a fully automated extraction of background content informationwithout requiring any additional information to be included. Thus, itensures a high degree of backwards compatibility. For example, thecontent analysis may indicate that the visual content signal is of afootball match which may be used to generate a surround imagecorresponding to e.g. spectators. The content analysis may for examplealternatively or additionally determine specific visual characteristicsof sections of the image of the primary display.

According to a different feature of the invention, the content analysiscomprises image object recognition. The content analysis mayspecifically detect the presence of objects in the visual contentsignal. For example, the content analysis may indicate that a section ofthe image of the primary display comprises e.g. clouds, grass etc andthis may be used to generate a suitable surround image.

According to a different feature of the invention, the means forgenerating a surround image is operable to perform motion estimation ofan image object and to generate the surround image in response to themotion estimation. This allows for a suitable way of enhancing thesurround image and to ensure increased continuity between the primarydisplay and the surround image. For example, the movement of an objectin the primary display may be tracked and continued in the surroundimage as the object reaches the edge of the primary display.

According to a different feature of the invention, the backgroundcontent information comprises a visual characteristic of an imagesection of the visual content signal proximal to an edge of the primarydisplay; the means for generating the surround image is operable togenerate at least a partial surround image having a corresponding visualcharacteristic; and the means for displaying the surround image isoperable to display the partial surround image proximal to the edge.

This allows for a high degree of continuity across an edge between theprimary display and the secondary display area and therefore facilitatesthat a combined display is achieved which may be perceived as a singlecontinuous image. Preferably, the content information comprises a visualcharacteristic of a plurality of image sections of the visual contentsignal proximal to different edges of the primary display, the means forgenerating the surround image is operable to generate a plurality ofpartial surround image sections which have a corresponding visualcharacteristic; and the means for displaying the surround image isoperable to display the partial surround images proximal to thecorresponding edges.

According to a different feature of the invention, the means forgenerating a surround image is operable to generate the surround imagein response to a predetermined image associated with the backgroundcontent information. This provides for a low complexity implementation.Specifically, the surround image or parts thereof may be generated byinclusion of a pre-stored image having similar visual characteristics.For example, a pre-stored image of a sky may be included in the surroundimage to extend an image section of the primary display comprising asky. The characteristics of the predetermined image may be modified toreflect the corresponding image section of the primary display. Forexample, the brightness and colour saturation may be modified to reflectthe image section.

According to a different feature of the invention, the means forgenerating the surround image is operable to generate the surround imagein response to a predetermined default image if no valid backgroundcontent information is determined. This allows for a neutral surroundimage to be presented if no appropriate background content informationcan be determined. The default image may specifically be a null surroundimage corresponding to no presentation of a surround image. This imagemay also be a colour, for example changing in brightness as a functionof the viewing angle (or distance from primary image).

According to a different feature of the invention, the means forgenerating the surround image is operable to generate the surround imageat a lower quality than a quality of the display of the content signalon the primary display.

The surround image or parts thereof may for example be of lowerresolution than the primary display. This facilitates processing and/orreduces the requirements of the functionality for generating thesurround image. Hence, for example, a reduced processing time or storagerequirement e.g. for predetermined images may be achieved.

According to a different feature of the invention, means for generatingthe surround image is operable to generate the surround image with aquality that decreases for increasing distance (viewing angle) from theprimary display. For example, the spatial resolution may be decreasingfor increasing distance (viewing angle). The inventor has realised thatthe sensitivity to reduced quality reduces with increasing distance fromthe primary display and that accordingly the quality may be increasedthereby reducing the complexity and cost of the involved equipmentand/or processing.

According to a different feature of the invention, the means forgenerating the surround image is operable to generate the surround imagein response to characteristics of a viewing environment associated withthe secondary display.

For example the projected surround image may depend on an ambientbrightness level in the viewing environment and/or the reflectivity andperhaps colour of any surfaces used for reflecting the surround image.Hence, an improved customisation for a specific viewing environment maybe achieved. The characteristics of the viewing environment may beautomatically determined or may for example be determined from a manualinput by a user.

According to a different feature of the invention, the visual contentsignal display apparatus further comprises means for determining acategory of the visual content signal and wherein the means forgenerating the surround image is operable to generate the surround imageprocessing in response to the category.

The category of the visual content signal may for example relate to agenre of the content of the visual content signal. For example,different algorithms and processes may be used for generating thesurround image depending on whether the content is action, sport, newsor music concert content. Hence, this allows for a surround image to begenerated which is optimised for the given category of visual contentsignal.

According to a second aspect of the invention, there is provided amethod of displaying a visual content signal comprising the steps of:receiving a visual content signal; presenting the visual content signalon a primary display; extracting background content information from thevisual content signal; generating a surround image in response to thebackground content information; and displaying the surround image on asecondary display area thereby providing a combined display having anincreased viewing angle.

These and other aspects, features and advantages of the invention willbe apparent from and elucidated with reference to the embodiment(s)described hereinafter.

An embodiment of the invention will be described, by way of exampleonly, with reference to the drawings, in which

FIG. 1 illustrates a visual content display apparatus in accordance withan embodiment of the invention;

FIG. 2 illustrates a viewing arrangement comprising a TV display inaccordance with prior art; and

FIG. 3 illustrates a viewing arrangement comprising a visual contentdisplay apparatus in accordance with an embodiment of the invention.

The following description focuses on an embodiment of the inventionapplicable to a visual content display apparatus particularly suited fora domestic consumer environment but it will be appreciated that theinvention is not limited to this application. For brevity, the termimage has in the description been used to include both single frameimages and multiple frame (e.g. moving) images and image sequences. Forexample, a surround image is not necessarily a single static image butmay include a (potentially unlimited) image sequence e.g. correspondingto a video clip.

FIG. 1 illustrates a visual content display apparatus 100 in accordancewith an embodiment of the invention.

The visual content display apparatus 100 comprises a receiver 101 whichreceives a visual content signal from a suitable source (not shown). Thesource may be external or internal and may for example by a TV broadcastsignal or a DVD disc. The receiver 101 comprises all necessaryfunctionality required for receiving the visual content signal and toextract or convert this into a suitable format. For example, for a TVbroadcast signal the receiver 101 comprises all required functionalityfor amplifying, filtering demodulating and decoding the received radiosignal to generate a base band visual content signal.

The visual content display apparatus 100 further comprises a primarydisplay processor 103 coupled to the receiver 101, and a primary display105 coupled to the primary display processor 103. The primary displayprocessor 103 is operable to process the visual content signal and tocontrol the primary display 105 to display the visual content signal. Inthe preferred embodiment, the receiver 101, primary display processor103 and primary display 105 are equivalent to a conventional TV set andare operable to receive a standard TV visual content signal and topresent in on the primary display 105.

In addition, the visual content display apparatus 100 comprises anextraction processor 107 which is coupled to the receiver 101 and whichreceives the base band visual content signal therefrom. The extractionprocessor 107 is operable to extract background content information fromthe visual content signal. The background content information relates tocharacteristics of the background of the image displayed on the primarydisplay.

In the preferred embodiment, the extraction processor 107 extractsbackground content information by extracting meta-data which has beenembedded in the visual content signal by the broadcaster. The meta-datacomprises information which is indicative of the background of the imageof the primary display. In the preferred embodiment, the backgroundcontent information relates to image sections of the picture which formsthe background of the scene displayed and thus of image sections whichtend to be relatively static during the scene. For example, for a newsbroadcast, the news reader will be considered to be foreground whereasthe remaining elements may be considered background. However, generallybackground content information may be considered relating to imagesegments which suitable can be extended and extrapolated beyond theedges of the primary display. When meta-data is used, the broadcastermay determine what is considered as background information by embeddingthe appropriate meta-data.

In the preferred embodiment, the meta-data relates to visualcharacteristics of the image of the primary display towards thedifferent edges of the primary display. For example, meta-data may beextracted indicating that the image sections towards the upper edge ofthe primary display corresponds to a sky, image sections towards thelower edge of the primary display corresponds to grass and imagesections towards the side edges of the primary display corresponds tograss and sky with a separation half way along the edge. Alternativelyor additionally, the meta-data may provide data indicating a preferredbackground colour or texture or other visual characteristic.

The extraction processor 107 is coupled to a secondary display processor109. The secondary display processor 109 is fed the extracted backgroundcontent information and is operable to generate a surround image inresponse to this. The secondary display processor 109 is in thepreferred embodiment coupled to a secondary display unit 111 which isoperable to display the surround image by displaying it on a secondarydisplay area which extends beyond the primary display 105.

In the preferred embodiment, the secondary display unit 111 comprises adisplay projector which is operable to project an image on an externalsurface. This surface may specifically be an external and preferablyportable reflecting screen but is in the preferred embodiment a surfaceof existing objects in the viewing environment. Preferably the projectoris operable to directly project the image on the walls, floor and/orceiling of the room in which the primary display is situated.

Hence, the embodiment provides for a system wherein a surround imageassociated with the background of a received and displayed image can beprojected on to a display area whereby the viewing angle may be extendedsignificantly and an enhanced and immersed viewing experience can beprovided.

To further illustrate the preferred embodiment, FIG. 2 illustrates aviewing arrangement comprising a TV display in accordance with priorart. FIG. 2 illustrates a room 201 having a first 203 and second 205 andthird 207 wall, a floor 209 and a ceiling 211. The room comprises aprimary display 213 which specifically may be a conventional flat screenTV mounted on the third wall 207.

The primary display 213 shows a film comprising a current image whichfor clarity and simplicity is illustrated as having four different imagesegments 215, 217, 219, 221. The first image segment 215 corresponds tothe background of the image towards the edge of the primary display 213which is closest to the first wall 203. The second image segment 217corresponds to the background of the image towards the edge of theprimary display 213 which is closest to the second wall 205. The thirdimage segment 219 corresponds to the background of the image towards theedge of the primary display 213 which is closest to the ceiling 211. Thefourth image segment 221 corresponds to the foreground of the image.

FIG. 2 illustrates the conventional viewing environment. A significantdisadvantage of this viewing arrangement is that the viewing experienceis limited by the small size of the primary display which not onlyreduces the impact of the displayed image due to the size of the imageitself but also allows for a high number of visual distractions. Forexample, a significant difference between the emotional experienceexperienced in a cinema and in a home cinema or television apparatus isdue to the much smaller viewing angle. Typically, a cinema screen is atleast an order of magnitude larger in size than that of domesticconsumer equipment. A wide viewing angle allows a viewer to be immersedin the scene and to be provided with more intense experiences.

FIG. 3 illustrates a viewing arrangement comprising a visual contentdisplay apparatus in accordance with the preferred embodiment of theinvention. Specifically, FIG. 3 illustrates a viewing experience thatmay be provided by the visual content display apparatus 100 of FIG. 1.

In the example, the extraction processor 107 has determined that animage section 215 exists close to the edge of the primary display 213extending towards the first wall 203. The characteristics of this imagesection are determined. For example, it may be determined that thisimage segment mainly comprises a green grassy background. Accordinglythe secondary display processor 109 controls the secondary display unit111 to project an image of green grass onto the first wall 203 and partof the third wall 207.

Likewise the second image segment 217 may also be determined as greenand grassy and accordingly the secondary display unit 111 may project animage of green grass onto the second wall 205 and part of the third wall207.

The extraction processor 107 may further determine that an image section213 exists close to the upper edge of the primary display 213. Thecharacteristics of this image section are determined as for examplemainly comprising a blue sky with clouds. Accordingly the secondarydisplay processor 109 controls the secondary display unit 111 to projectan image of a cloud onto the ceiling 211 and part of the third wall 207.

Thus a viewer will in this case be presented with a partiallysurrounding image which has a much increased viewing angle compared tothe original display. The surround image projected on to the walls andceiling need not be an accurate match for the background of the image ofthe primary display or have a smooth and unnoticeable transition fromthe image of the primary display. Rather an improved viewer experiencecan be achieved even with rough approximations to the primary displayimage and in particular for some images a projection of associatedcolours may be sufficient for an enhanced experience.

As a specific example, if the primary display shows a person talking inthe middle of a populated square of a city, buildings and other similarfeatures may be projected onto the side walls, sky and clouds may beprojected on the ceiling and pictures of streets with coloured mosaicmay be projected onto the floor. In this way, the viewer may still seethe main focused image on the primary display but will also, perhaps outof the corner of his eyes, see the facades of the houses at the sidesand the clouds in the sky. In this way a surround image is created thatprovides a much more intense viewing experience.

The images projected may be derived from the different image sectionsfor example by a suitable repetition, blurring and resampling of theimage data of these image sections. Alternatively or additionally,existing pre-stored images may be used. For example, the visual contentdisplay apparatus may comprise a large number of pre-stored imagescorresponding to different possible background content characteristics.An image (including an image sequence) may e.g. be stored correspondingto a slightly cloudy sky, another image for a more clouded sky, anotherimage for a sky with no clouds etc. Accordingly, the extractedbackground content information may be used to select a suitable imagefor a given image section and this may be projected onto thecorresponding part of the display area.

In one embodiment, meta-data may thus be extracted from the visualcontent signal and used to select a suitable pre-stored image. Thisimage may have characteristics amended to correspond to e.g. thebrightness and colour saturation of the image of the primary display.The image may subsequently be merged with other images selected forother image segments and the combined surround image may be projectedonto the walls and/or ceiling and/or floor of the room. Specifically, insome embodiments, the meta-data may directly indicate or identify asuitable image intended to be used with the visual content signal.

If no suitable background content information can be determined orequivalently if no surround image can be generated on the basis of thedetermined background content information, the visual content displayapparatus preferably projects a default image. The default image mayspecifically be a null image corresponding to not projecting any image.However, in the preferred embodiment, the default image is an imagewhich frames the primary display to produce a visually appealingdisplay.

The secondary display unit 111 may preferably use a projector/beamercomprising a rotating polygon whereby images can be projected at severalsurfaces in the room. The projector can be placed behind the viewer, infront of the primary display, on the ceiling or in any suitablelocation. In some embodiments, a plurality of projectors may be used.

In the preferred embodiment, the processing and generation of thesurround image is in real time. In this embodiment, background contentinformation is extracted in real time from the visual content signal asand when it is received and this information is used to generate asurround image that is synchronous with the primary display. Preferably,any delay associated with the processing and generation of the surroundimage is compensated by the visual content signal being delayedequivalently for the primary display.

In some embodiments, background content information from previouslyshown images may be used to generate the surround image corresponding toa current image. This is particularly advantageous in embodiments whereit is preferable not to delay the visual content signal of the primarydisplay.

In some embodiments, the background content information may be extractedby content analysis of the visual content signal.

Content analysis may be based on detecting specific characteristicstypical for a given category of content. For example, a video contentitem may be detected as relating to a football match by having a highaverage concentration of green colour and a frequent sideways motion.Cartoons are characterised by typically having strong primary colours, ahigh level of brightness and sharp colour transitions.

Thus video coding parameters may advantageously be used to determine thecontent of a video signal. For example, a high relative value of ACcoefficients in a DCT transform block indicates that a sharp transitionis likely to be comprised in the transform block. Such a transition istypical for a cartoon and may therefore be included as a video codingparameter that indicates that the current content is a cartoon.Typically, a significant number of parameters are considered and thecontent may be determined as the content category which most closelycorrelates with the determined characteristics. Thus, the coloursaturation and luminance may further be included to determine if thecurrent content is a cartoon. For example, if video coding dataindicates a high degree of colour saturation, high luminance, a highconcentration of energy in high frequency DCT coefficients as well aslarge uniform or flat picture areas, a content analysis algorithm maydetermine the current content as a cartoon.

Typically, both video coding parameters and non-video coding parametersmay be used together for content analysis. For example, a high degree ofmotion, strong luminance and a rhythmic nature of an associated soundtrack may indicate that the current content is a music video.

Further information on content analysis is generally available to theperson skilled in the art. For example, the articles “Content-BasedMultimedia Indexing and Retrieval” by C. Djeraba, IEEE Multimedia,April-June 2002, Institute of Electrical and Electronic Engineers; “ASurvey on Content-Based Retrieval for Multimedia Databases” by A.Yoshika et al., IEEE Transactions on Knowledge and Data Engineering,vol. 11, No. 1, January/February 1999, Institute of Electrical andElectronic Engineers; “Applications of Video-Content Analysis andRetrieval” by N. Dimitrova et al., IEEE Multimedia, July-September 2002,Institute of Electrical and Electronic Engineers and the thereinincluded references provide an introduction to content analysis.

Thus in some embodiments, the determination of content may be used todetermine estimates of the background content information for a givenvisual content signal. For example, if it is determined that the visualcontent signal relates to a football match a surround image comprisinge.g. spectators or flags associated with the teams may be generated. Ifthe category is determined as cartoons, strong primary colours may beused in the surround image.

In some embodiments, a simplified content analysis may be performedwhich simply determines visual characteristics of different imagesegments. For example, a content analysis may be performed to determinethe contents of the image. Hence, for example object identification orrecognition of image objects may be used to generate background contentinformation. For example, an object detection algorithm may determinethat the image comprises a moving object. Preferably, the objectdetection is combined with motion estimation and the surround image isgenerated in response to the estimated motion.

For example, a moving object may be detected and the trajectory of theobject in the primary display may be determined. As the object reachesthe edge of the primary display, the visual content display apparatusmay extrapolate the trajectory and display the continued movement of theobject into the surround image.

As further examples, the content analysis and processing may comprise afiltering of the background from the foreground. For example objectdetection and e.g. pattern recognition may be used to detect objectswhich can be separated from the background. Examples of parameters thatmay be determined by the content analysis or processing and which can beused in generating the surround image include the average colour of thebackground in the image; the intensity or brightness of the image; theduration and frequency of different scenes and dynamic variations inindividual clips or scenes.

In the preferred embodiment, the surround image is generated at a lowerquality level than that of the primary display. The inventor hasrealised that e.g. a high spatial resolution is only required for theprimary display which is the centre of focus of the viewer. The viewertypically perceives the sharp contours and colours of the image of theprimary device (the actual point of focus) while the perception of thebackground is less sharp (only the coarse contours and colours areperceived at the wider viewing angles).

Accordingly, in the preferred embodiment, the surround image is at alower spatial resolution than that of the primary display. Indeed, inthe preferred embodiment the quality and in particular the resolution ofthe surround image reduces for increasing distance from the primarydisplay (increasing viewing angle). The reduction does not need to bemonotonic but may for example be a step function of the distance to theprimary display.

A reduced quality level may for example allow for less computationaltime being required for extraction of the background content informationas less accurate information (lower spatial and dynamic resolution) isrequired. Furthermore, less storage capacity is required for storing anypre-stored background images. Additionally the equipment for projectingthe surround image can be less complex and thus cheaper.

In the preferred embodiment, the projected surround image may further becustomised to the specific viewing arrangement and content of the visualcontent signal.

In the preferred embodiment, the visual content display apparatus isoperable to modify the processing parameters and algorithms used forextracting the background content information and for generating thesurround image depending on the category (e.g. genre) of the visualcontent signal. For example, the content that may be enhanced bysurround images includes movies/soaps, music concerts, sport games,holiday pictures, documentaries, theatre/plays, animal series andscience fiction programmes. For example, a space science fiction scenemay be enhanced by a surround image showing approaching space ships andstars on the ceiling.

The processing and the nature of the surround image may depend on thegenre of content of the visual content signal. For example, an algorithmthat produces a night sky with fireworks may be generated if the contentcorresponds to an outdoor rock concert whereas a night sky with distantstars and spaceships may be generated if the content corresponds to anoutdoor scene at a space station in a science fiction programme.

In the preferred embodiment, the surround image is furthermore generatedin response to characteristics of the viewing environment. As a simpleexample, the surround image is preferably projected such that itprovides a brightness less than the primary display but sufficientlybright for clear viewing. As the viewed image will depend on thereflectivity of the surfaces projected on to, as well as the generalbrightness level in the room, the brightness of the projected imagedepends on the brightness level and the nature of the surfaces used.

These and other features may for example be detected by sensorsincluding for example a light intensity meter.

Other examples of features that may allow a further customisation andthus enhancing the viewing experience include the following:

The visual content display apparatus may comprise an input for modifyingthe intensity of the surround image and thus the intensity of theexperience. A sensitive person may prefer a slight enhancement whereas aviewer more familiar with the display apparatus may prefer a moreenhanced experience. Furthermore, in some embodiments, the visualcontent signal display apparatus may comprise means for determining acategory of the visual content signal and the means for generating thesurround image may be operable to generate the surround image inresponse to the preferred sensitivity as determined by the user.

The visual content display apparatus may comprise an input for changingthe dynamics of the surround image.

The visual content display apparatus may comprise a switch to choose thesurround image being based on meta-data or image processing/analysis.

The visual content display apparatus may comprise means for determininga user profile reflecting preferred colours and dynamics of the surroundimage (e.g. rate of switching, colour preference, etc.). For example, ifa user has seen a movie four times, a different viewing experience maybe desirable at the fifth viewing. In that case, the user profile may bestored and used to control the settings (e.g. a sensitivity level) ofthe visual content display apparatus for future viewings.

The visual content display apparatus may provide the surround imageselectively. For example, the surround image may be provided only whenpredefined events occur (for example, only when a goal is scored in afootball match).

In some embodiments, the surround image may be determined partly inresponse to processing of transmitted audio and video signals receivedby sensors, such as a microphone, CCD camera's, etc. These signals maybe used in addition to the extraction of information from the visualcontent signal and/or may be comprised in the visual content signal asan embedded data stream and extracted therefrom. The information thatcan be extracted from the sensor signals may be similar to thatextracted from the visual content of the visual content signal. Inaddition, environmental aspects may be taken into account, such as thelight intensity in the room (day or night, light sources in room), thenumber of persons in the room, voice recognition to process orders fromuser (switch to a different genre, or increase the sensitivity,brightness of surround images), adaptation of surround image tohappenings (like parties, conversation at background, etc.),self-learning etc.

As a specific example, a visual content display apparatus may beprovided as a consumer surround image control box which contains anumber of features and control means including for example thefollowing:

-   -   Control input for changing the intensity of the surround        experience.    -   Control input for selecting a content genre.    -   Control input for selecting a content item category    -   Control input for changing the dynamics of the surround image        processing variations.    -   Control input for controlling a scene contrast.    -   Control input or automatic means for selecting and/or storing a        user profile.    -   Means for entering a self-learning mode (e.g. measuring or        determining characteristics of the operations of the surround        image control box such as the number of played scenes, for        example succeeding goals in a soccer game can be emphasized in        time).    -   Polarization control means: e.g. for controlling that        enhancement occurs only for predefined events.    -   A Source selector for selecting source information for the        surround image processing such as e.g. which information from an        audio/video content signal to use (color, intensity, image        recognition, etc.).    -   Control input for controlling an adaptation to the current        viewing environment.    -   A purpose selector for selecting e.g. a purpose of the surround        experience thereby allowing the surround images to be selected        to most suitably achieve this purpose.    -   A mood selector.

The invention can be implemented in any suitable form includinghardware, software, firmware or any combination of these. However,preferably, the invention is implemented at least partly as computersoftware running on one or more data processors and/or digital signalprocessors. The elements and components of an embodiment of theinvention may be physically, functionally and logically implemented inany suitable way. Indeed the functionality may be implemented in asingle unit, in a plurality of units or as part of other functionalunits. As such, the invention may be implemented in a single unit or maybe physically and functionally distributed between different units andprocessors.

Although the present invention has been described in connection with thepreferred embodiment, it is not intended to be limited to the specificform set forth herein. Rather, the scope of the present invention islimited only by the accompanying claims. In the claims, the termcomprising does not exclude the presence of other elements or steps.Furthermore, although individually listed, a plurality of means,elements or method steps may be implemented by e.g. a single unit orprocessor. Additionally, although individual features may be included indifferent claims, these may possibly be advantageously combined, and theinclusion in different claims does not imply that a combination offeatures is no feasible and/or advantageous. In addition, singularreferences do not exclude a plurality. Thus references to “a”, “an”,“first”, “second” etc do not preclude a plurality.

1. A visual content signal display apparatus comprising: means forreceiving a visual content signal; means for presenting the visualcontent signal on a primary display; means for extracting backgroundcontent information from the visual content signal; means for generatinga surround image in response to the background content information; andmeans for displaying the surround image on a secondary display areathereby providing a combined display having an increased viewing angle.2. The visual content signal display apparatus of claim 1 wherein themeans for displaying the surround image is operable to project thesurround image on to an external surface of an object.
 3. The visualcontent signal display apparatus of claim 2 wherein the external surfaceis an internal surface of a room.
 4. The visual content signal displayapparatus of claim 1 wherein the means for extracting is operable toextract real time background content information from the visual contentsignal, and the means for generating the surround image is operable togenerate a real time surround image in response to the real timebackground content information.
 5. The visual content signal displayapparatus of claim 1 wherein the means for extracting is operable toextract the background content information in response to backgroundmeta-data comprised in the visual content signal.
 6. The visual contentsignal display apparatus of claim 1 wherein the means for extracting isoperable to extract the background content information in response to acontent analysis of the visual content signal.
 7. The visual contentsignal display apparatus of claim 6 wherein the content analysiscomprises image object recognition.
 8. The visual content signal displayapparatus of claim 7 wherein the means for generating a surround imageis operable to perform motion estimation of an image object and togenerate the surround image in response to the motion estimation.
 9. Thevisual content signal display apparatus of claim 1 wherein thebackground content information comprises a visual characteristic of animage section of the visual content signal proximal to an edge of theprimary display; the means for generating the surround image is operableto generate at least a partial surround image having a correspondingvisual characteristic; and the means for displaying the surround imageis operable to display the partial surround image proximal to the edge.10. The visual content signal display apparatus of claim 1 wherein themeans for generating a surround image is operable to generate thesurround image in response to a predetermined image associated with thebackground content information.
 11. The visual content signal displayapparatus of claim 10 wherein the means for generating the surroundimage is operable to generate the surround image in response to apredetermined default image if no valid background content informationis determined.
 12. The visual content signal display apparatus of claim1 wherein the means for generating the surround image is operable togenerate the surround image at a lower quality than a quality of thedisplay of the content signal on the primary display.
 13. The visualcontent signal display apparatus of claim 1 wherein the means forgenerating the surround image is operable to generate the surround imagewith a quality that decreases for increasing distance from the primarydisplay.
 14. The visual content signal display apparatus of claim 1wherein the means for generating the surround image is operable togenerate the surround image in response to characteristics of a viewingenvironment associated with the secondary display area.
 15. The visualcontent signal display apparatus of claim 1 further comprising means fordetermining a category of the visual content signal and wherein themeans for generating the surround image is operable to generate thesurround image processing in response to the category.
 16. A method ofdisplaying a visual content signal with a visual content displayapparatus, the method comprising: receiving a visual content signal witha receiver of the apparatus; presenting the visual content signal on aprimary display; extracting background content information from thevisual content signal with an extraction processor of the apparatus;generating a surround image in response to the background contentinformation with a secondary display processor of the apparatus; anddisplaying the surround image on a secondary display area therebyproviding a combined display having an increased viewing angle.
 17. Themethod of claim 16, further comprising storing a plurality of imagescorresponding to a plurality of background content characteristics, andwherein generating a surround image in response to the backgroundcontent information includes selecting one of the stored images inresponse to the extracted background content information.
 18. A systemcomprising: a receiver for receiving a visual content signal; a primarydisplay processor for processing the visual content signal and forproviding an image for display on a primary display; an extractionprocessor for extracting background content information from the visualcontent signal; a second display processor for generating a surroundimage in response to the extracted background content information; and asecondary display for displaying the surround image on a secondarydisplay area surrounding the primary display, wherein a combination ofthe image and the surround image provide a greater viewing angle thanthe image alone.
 19. The system of claim 18, wherein the system stores aplurality of images corresponding to a plurality of background contentcharacteristics, and wherein the second display processor selects one ofthe stored images in response to the extracted background contentinformation and generates the surround image from the selected storedimage.
 20. The system of claim 18, wherein the system stores a pluralityof images corresponding to a plurality of background contentcharacteristics, and wherein the second display processor selects atleast two of the stored images in response to the extracted backgroundcontent information and generates the surround image from the at leasttwo selected stored images.